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The Influence of Designer and Contextual Variables on the Incorporation of 
Motivational Components to Instructional Design and the Perceived Success of a 

Project 

Bonnie J. Shellnut 

Wayne State University , Detroit , Michigan 



Abstract 

This study investigated designer background characteristics and contextual variables that were 
hypothesized to influence the incorporation of motivational components to the design of instructional 
materials as well as the factors that may affect the perceived level of a project’s success. A hypothesized 
model was developed that depicted anticipated relationships (based on the literature) between the variable 
sets on the perceived level of success of a project. Of the 500 questionnaires mailed to instructional design 
practitioners, 201 valid responses were returned. Stepwise multiple linear regression analyses and path 
analysis techniques were used to answer the 11 research questions posed for this study and test the 
hypothesized model. The revised model was developed from the results of stepwise regression analyses and 
path analysis techniques. The study found that certain background and contextual variable influence the 
incorporation of motivation design components and that certain factors in the hypothesized model have a 
significant relationship to the perceived level of success of a project. 

Problem Statement 

Is motivation an important component to the design of instructional materials? Most theorists and 
practitioners would agree that motivation should be a part of every design. However, scholars note that research on 
learner motivation has lagged far behind research on the learner’s cognitive characteristics (Herndon, 1987; Klein & 
Keller, 1990; Spitzer, 1996). Traditionally the design process have focused on analysis of the learner’s cognitive 
characteristics. According to Richey, (1992) “learner interests are the most common attitudes addressed in ISD 
models” (p. 1 16). Much in the literature indicates that while people accept that motivation is a desirable aspect of 
instruction it is considered too vague and complex a concept to be applied to systematic instructional design (Farmer, 
1989; Keller, 1987c; Wlodkowski, 1981). 

This study investigated designer background characteristics and contextual variables that were hypothesized 
to influence the incorporation of motivational components to the design of instructional materials as well as the 
factors that may affect the perceived level of a project’s success. However, the emphasis in the literature regarding 
background characteristics and contextual variables is primarily concerned with the influence each may have on the 
effectiveness of the instruction for the learners. There is little research related to the possible influences of 
background characteristics, attitudes, and use of instructional design elements on a designer’s incorporation of 
motivational strategies in to the materials and on his or her perceived success of a project. 

Traditional Instructional Systems Design (ISD) models generally follow a systematic approach to design 
and are based on learning theory. They usually prescribe conducting a learner analysis to determine knowledge and 
skills during the definition or analysis phase. Learner characteristics such as age (Bohlin, Milheim, & Viechnicki, 
1990; Bohlin & Milheim, 1994; Richey, 1992; Wlodkowski, 1985); gender (Binns & Branch, 1994; Canada & 
Brusca, 1991), culture, ethnicity (Eastman & Smith, 1991; Powell, 1997), organizational context (Tessmer & Richey, 
1997) are considered important elements of a thorough audience analysis. Theorists and practitioners agree that 
background characteristics may have a significant effect on learner motivation or learning itself. However, analysis 
of designers’ knowledge and skills is a relatively new subject in the literature. Little is said about how background 
and contextual factors may affect designers as they plan instruction and determine instructional and motivational 
strategies. 

Keller (1983b) defined motivation as “the magnitude and direction of behavior. ... it refers to the choices 
people make ... and the degree of effort they will exert. Motivation is influenced by a myriad of internal and external 
characteristics (p. 389). He (Keller, 1987a) developed the ARCS Model of Motivational Design from his theoretical 
model (1979), which was based on accepted theories of human motivation. ARCS is an acronym for attention, 
relevance, confidence, and satisfaction, the four components of the model. Keller (1987b) defined each component 
as follows: 



• Attention - Capturing the interest of learners; stimulating the curiosity to learn. 

• Relevance - Meeting the personal need/goals of the learner to affect a positive attitude. 

• Confidence - Helping the learners believe/feel that they will succeed and control their success. 

• Satisfaction-Reinforcing accomplishment with rewards (internal and external) (p.2) 
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Methods and Procedures 



A hypothesized model was developed, based on a review of the literature, that identified variables and their 
subcomponents and the possible relationships between and among them. The variables sets used in the hypothesized 
model were (a) designer background, (b) instructional context, (c) attitude toward motivational design, (d) use of 
ISD components, (e) incorporation of motivation design components, and (f) perceived level of the project’s success. 
Each of these sets had subcomponents, which constituted the variables. The primary hypothesis was that one or more 
of these variables would have a correlative relationship with the designer’s application of motivational components 
of attention; relevance, confidence, and satisfaction to the design process. A second hypothesis was that 
incorporation of motivation would be shown to have an impact on the perceived success of the project. (See Figure 
1 ) 

The survey instrument was an 81-item questionnaire with five sections mapped to the variable sets in the 
hypothesized model was designed and developed by the researcher. Before finalizing the questionnaire, it was given 
to 25 members of the Michigan Chapter of ISPI (International Society for Performance Improvement) The 
questionnaire was mailed to 500 members of ISPI throughout the 50 United States and Puerto Rico. There were 201 
usable questionnaires, which constituted a 40% response rate. Descriptive statistics were reported to provide a 
profile of the respondents. 

The characteristics in 201 respondents included the following: a fairly even representation of men (n=91, 
45%) and women (n=110, 56%). Nearly all of the participants were Caucasian (n=186, 93%) with few (n=15, 7%) 
representing other ethnic groups. Most respondents (n=145, 72%) had a master’s degree or above. The majority of 
the sample (n= 130, 65%) were more than 40 years old, and had an average of 13 years of design experience. Nearly 
all respondents (n=156, 79%) rated their design expertise as high or very high, but only 76 (38.5%) claimed to have 
high or very high expertise in motivational design. 

The data analyses answered 11 research questions. The hypothesized model (See Figure 1) depicted 
expected relationships between the following variables: 

1 . Designer background and attitude toward motivation. 

2. Instructional context and attitude toward motivation. 

3. Designer background and use of cores ISD elements. 

4. Instructional context and use of core ISD elements. 

5. Designer background and incorporation of motivation design components, (analysis, attention, relevance, confidence, and 
satisfaction). 

6. Instnictional context and the incorporation of motivation design components, (analysis, attention, relevance, confidence, 
and satisfaction). 

7. Designer attitudes toward motivation and the incorporation of motivation and its components, (analysis, attention, 
relevance, confidence, and satisfaction). 1 

8. Designer use of core ISD elements and the incorporation and the incorporation of motivation components, (analysis, 
attention, relevance, confidence, and satisfaction). 

9. Four variables — designer background, instructional context, designer use of core ISD elements and designer attitude 
toward motivation — and the incorporation of motivation and its components (analysis, attention, relevance, confidence, 
and satisfaction). 

10. The incorporation of motivation and its components, (analysis, attention, relevance, confidence, and satisfaction). 

11. All five variables: designer background, instructional context, designer use of core ISD elements and designer attitude 
toward motivation, and the incorporation of motivation and its components (analysis, attention, relevance, confidence, and 
satisfaction) and the perceived level of a project’s success. 



Limitations of the Study 

Some recognized limitations are involved in survey studies. First, despite a mailed questionnaire’s ability to 
reach large numbers of people, the response rate is often low (Knupfer & McLellan, 1996), which may make it 
difficult to obtain a representative sample. Second, the mailed questionnaire is limited to volunteers, who may be 
different from non-respondents (Knupfer & McLellan, 1996). Third, if the sample is drawn from members of a 
professional organization, the sample has a possibility of being skewed toward more highly trained practitioners than 
the norm. 
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Figure I. Hypothesized Model of Variables Related to the Incorporation of Motivational Components to 
Instructional Design and the Perceived Level of the Project's Success 




Stepwise multiple linear regression analyses and path analysis techniques were used to answer the 1 1 
research questions posed for this study and test the hypothesized model. The revised model was developed from the 
results of stepwise regression analyses and path analysis techniques. To determine the relationships between the 
variable sets, regression analyses and path analysis were used. The results of these analyses are depicted in the 
revised model. (See Figure 2.) 

Discussion of Findings 

The revised model depicts the variables that were determined to be valid predictors of the dependent 
variable - perceived success of the project. One the five original variable sets, designer attitudes toward motivation, 
in the hypothesized model was eliminated. Four of the variable sets, (a) designer background, (b) instructional 
context, (c) use of core ISD elements, and (d) incorporation of motivation design components, were found to be 
valid predictors. Eight variables or subcomponents from the four variable sets were shown to have a direct effect on 
the dependent variable, perceived level of the project’s success. 
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Figure 2. Revised Model of Variables Related to the Incorporation of Motivational Components to 
Instructional Design and the Perceived Level of the Project ’s Success 
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Designer Background 

Respondents rated their general design expertise and motivation expertise. Designer expertise was not 
defined in the questionnaire. However, designer expertise has been defined in the literature as years of experience in 
the field as well as the ability to internalize and apply theoretical principles of instructional systems design 
(Atchinson, 1996; Rowland, 1992). 

Motivation expertise was the only variable from designer background characteristics that directly affected 
perceived success of the project. This finding appeared to be consistent with McCombs’ (1986) conclusion that 
expertise in design is one of the critical factors leading to successful application of an ISD model. Finding that 
motivation expertise has a direct effect on the perceived success of the project suggests that increased experience 
with motivational design increases the possibilities for a project’s perceived success. Motivation expertise also had 
an indirect effect on the perceived success of the project through the confidence and satisfaction components of 
motivational design. The confidence component in motivational design emphasizes using strategies to guild a 
positive expectation for success, enhancing the student’s beliefs in their competence, and ensuring personal control 
of learning (Keller, 1987c). The satisfaction component in motivational design emphasizes helping learners feel 
positive about their achievement. 

Confidence-building strategies may be applied to the sequencing of content such as ordering problems from 
easy to hard to help ensure success, strategy applied successfully to computer courseware design (Keller & Suzuki, 
1988). Additionally, strategies to increase confidence may be used to influence learners’ attitudes. Motivational 
messages were used effectively to increase positive attitudes and learner confidence regarding the completion of a 
difficult course conducted in a summer workshop for teachers in Mozambique (Visser & Keller, 1990). 

The influence of motivation expertise on the selection of confidence and satisfaction strategies, however, 
conflicts with Farmer’s (1989) research that showed satisfaction to be the least addressed motivation component by 
designers. One explanation for this difference may be that the designers in Farmer’s study involved novice designers 
who were graduate students in instructional technology. The designers in the present study had a mean of 13 years 
experience. This suggests that experienced designers with practice in the design of motivation are more likely to 
incorporate confidence and satisfaction strategies from the ARCS Model. 

These findings pertaining to confidence and satisfaction are not consistent with research by Means, 
Jonassen, and Dwyer (1997), who found that relevance was the primary component that influenced motivation and 
learning outcomes. Although designers in the present study cited relevance most often as the key element in 
motivation, relevance did not appear as a predictor of project success. These contradictory findings about motivation 
are consistent with Farmer’s (1989) conclusions about designers’ decisions. He found that designers were more 
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likely to use “their personal intuitive feelings about what was motivating” rather than follow recommended strategies 
suggested by the ARCS Model (p.189). Furthermore, Farmer noted that designers’ reluctance to incorporate 
motivation into instruction systematically stemmed from their attitude about motivation. He observed that most 
subjects in his study viewed motivation as ‘fuzzy and uncertain” (p. 190). 

The literature is inconclusive about the effectiveness of applying motivational design strategies to achieve 
desired instructional goals (Means et al., 1997). Some studies have found positive results, while others have found no 
significant effects (Moller, 1994; Naime-Deifenbach, 1991). However, one explanation for varying conclusions may 
be that each study was different in its goals, samples, and procedures. Therefore, more research, especially 
replication studies, regarding the uses and effectiveness of motivational design is needed. 

Instructional Context. 

Recent research into the role of context suggests that contextual variables play a role in successful training 
and learning (Tessmer, 1995; Tessmer & Richey, 1997). However, there is little information about the possible 
relationship between where the designers’ products are used (own organization or client’s organizations) designers 
and the perceived success of a project. 

Four variables from the instructional context variable set also directly affected the perceived level of the 
project’s success. These contextual variables were: (a) place products used: another organization, (b) classroom/ 
instructor-led delivery, (c) amount of facilities allocated, and (d) importance of learners’ attitudes 

The first predictor was the place products were used. The respondents indicated where their products of 
their work were typically used— in their own organizations or in a client’s organization. This study found a positive 
relationship between use of the designers’ instructional products in the clients’ organizations and perceived success. 

It was not clear why this why this was so. One possible explanation may be due to the type of evaluations general 
done on instructional products. Designers and trainers commonly use “reaction,” or “smile sheets,” to obtain a first 
level evaluation of the instruction (Kirkpatrick, 1994). These evaluations are often constructed by the 
designer/trainer, and are scored and interpreted by the designer/trainer. Therefore, a limitation of this study is that the 
respondents were not asked s to identify the types of evaluation methods they used to reach their conclusion about 
the success of the project. Further, there were no objective data collected from the learners or clients on the success 
of the project. The determination of the project’s success depended entirely on the respondents’ perceptions. 

Classroom/instructor led delivery was the second predictor of perceived project success. The positive 
relationship between the two variables indicated that respondents who reported greater use of classroom delivery 
tended to rate the project’s success higher. Classroom delivery/instructor led delivery also indirectly influenced 
perceived level of project success through confidence and satisfaction. Theses positive relationships may be due to 
the fact that most of the respondents (n=166, 83%) cited instructor-led classroom instruction as the delivery type 
used in the selected project. Although respondents could have indicated more than one type, other delivery types 
such as print (n=69, 34.5%), computer (n=63, 31.5$), video (n=26, 13%) were not used as often and did not appear 
as significant predictors. 

The third contextual variable found to be a significant predictor of perceived project success was the 
amount of facilities allocated. However, this was a negative relationship, which suggested that participants who 
estimated the amount of facilities for the projects as low or moderate were more likely to rate the success of the 
projects higher. This finding seems to contradict research by Tessmer and Richey (1997), who argue: 

The physical condition of the immediate environment is a potent force in learning. The physical 
environment does not so much increase learning when it is excellent as inhibit it when it is poor. That is, a certain 
level of adequacy must be attained in seating, acoustics, temperature, and lighting for proper learning to take place 
(P-97). 

However, this finding may not be completely in contrast to Tessmer and Richey’s contextual analysis. 
Respondents’ rated the “amount of facilities.” as low or moderate, but this may indicate that they perceived them to 
be “adequate.” As noted previously, more than 80% of the participants in this study indicated that their project 
involved classroom instruction with approximately one third using video and or computer facilities. The “low” to 
“moderate” classroom facilities used for classroom instruction by the participants, then, was deemed sufficient for 
the perceived level of project success. This finding may indicate that the amount of facilities allocated is not a 
critical element to successful instructor- led classes that are not dependent on superior facilities and equipment. 

Importance of learners’ attitudes was the fourth contextual variable to directly predict the perceived level of 
a project’s success. This finding indicates that respondents who reported being concerned about learners’ attitudes 
were likely to perceive the project as being successful. This finding is supported by the literature. Richey (1992) 
notes that ISD models generally address learner interests, which are related to attitudes. A leading instructional 
design textbook (Rothwell & Kazanas, 1992) recommends that a needs analysis include attitudes. Since Keller 
(1979) and others (Bohlin, Milheim, & Viechnicki, 1991; Bohlin et al., 1990; Keller, 1987b; Keller, 1990; Richey, 
1992; Wlodkowski, 1985) consider learners’ attitudes to be connected to motivation, it is important to find that 
practicing designers also consider attitudes of learners to be important. 



Use of Core ISO Elements 

Respondents rated the importance core ISD elements were to the projects identified for this study. The 
importance of using an ISD model (or variable “use a model”) was found to be a negative predictor of the perceived 
level of the project’s success. This finding indicated that importance of using an ISD model does not have a 
significant relationship to the perceived success of the project. This finding was not consistent with the theory of 
ISD, which endorses the use of an ISD model for the design and development of instructional materials. This study’s 
finding, however, supports research on designers and the difference between designer practice and theory (Pieters & 
Bergman, 1995; Rowland, 1993; Winer & V&zquez-Abad, 1995). This finding underscores the argument by Pieters 
and Bergman, who note that . .design is often tacit (i.e., done without knowledge that a systematic procedure is 
being followed)” p.123. 

The importance of using an instructional design model (“use a model”) had an indirect effect through the 
motivation component, confidence. This effect was positive, indicating that instructional designers who considered 
the use of an ISD model important were more likely to include confidence-building strategies into the design of 
instructional materials. 

Incorporation of Motivation Design Components. 

The hypothesized model contained five subcomponents in the variable set. Incorporation of Motivation 
Design Components. Based on the ARCS Model (Keller, 1987c), this variable set included the following 
components: (a) motivation analysis, (b) attention, (c) relevance, (d) confidence, and (e) satisfaction. According to 
comments made by the respondents, effective instructional design includes the use of motivational strategies. 
However, the revised model showed that only confidence and satisfaction were directly related to the perceived level 
of the project’s success. Both of these variables were positive, which indicated that respondents who indicated they 
emphasized confidence and satisfaction in their designs were likely be rate the success of the project as high. It is not 
clear why these two strategies emerged as predictors but the others did not. An explanation may be in the 
contradiction that exists between theory or beliefs and actual practice. The majority of respondents (121, 60%) 
agreed with the statement that motivation can be analyzed effectively and applied systematically to instructional 
design. Nonetheless, designers in this study appeared less likely to analyze motivation, but seemed likely to include 
one or more of motivation components (attention, relevance, confidence, and satisfaction) when planning, 
developing, and implementing their projects. This lack of motivational analysis supported the general view in the 
field that motivation is somewhat of a vague concept that is difficult to be analyzed systematically (Farmer, 1989; 
Keller, 1987c). In addition, this finding showing that respondents placed more importance on confidence and 
satisfaction components than attention and relevance may be in accordance with Keller’s (Keller, 1987b) 
recommendation for the selection of components based on a learner analysis of motivational needs. 

Conclusions: Implications for Practice and Recommendations for Research 

The initial problem statement presented in this study suggested that one or more of the variables in the 
hypothesized model would affect a designer’s application of motivational components to the design process and 
ultimately a designer’s perceived success of the project. The relationships between these variables were analyzed. 

Because this study surveyed members of ISPI who are involved in design activities, the results of their 
observations should provide several implications for the field. 

1. Motivation expertise is the only one of the designer background characteristics that influences the designer’s perceived 
success of the project. This suggests that designer’s should make an effor to incorporate motivational design strategies to 
increase the likelihood of a successful project. 

2. Background characteristics such as age, gender, race/ethnicity, and education level do not appear to be factors that 
influence the designer’ incorporation of motivation nor the designer’s perceived success of the project. While these factors 
may be important considerations for a learner analysis regarding the design of motivational strategies, these do not appear 
to have any relationship to the designer’s incorporation of motivational components or the success of the project. 

3. Instructional context variables regarding the importance placed on learners’ attitudes was shown to have a significant 
relationship to the project. Designers should make sure that these learner characteristics are given significant consideration 
in the design of instructional materials. 

4. Classroom or instructor led delivery is another contextual variable that affects the perceived success of a project. This 
suggests that instruction delivered via a classroom with an instructor has increased likelihood for success. 

5. The importance of using of an ISD model in the design of instructional materials had a negative relationship to the 
perceived success of the project. This suggests that designers probably consider the reaction to the end product as more 
important than the process used to create the end product. Further, this may indicate that experienced designers apply tacit 
knowledge to the design process and incorporate various steps from different models rather than selecting just one model. 

Recommendations for Further Research 

The findings indicate that certain designer background and contextual variables have significant 
relationships to the incorporation of motivational components and to the perceived success of a project. The data 
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show motivation expertise is an important factor in predicting the success of a project and that certain contextual 
factors and design practice variables influences the outcome of a project. However, there are several issues that need 
further research. 

Expertise had a positive impact on the perceived success of a project and the incorporation of motivation 
analysis. However, this study did not define or describe motivation expertise. Rowland studied instructional design 
expertise but motivational design expertise has not been a subject of research. More needs to be done to describe and 
define the practice of motivational design. 

The two motivational components found to have the most impact on perceived success of the project are 
confidence and satisfaction. A study by Means and his colleagues show relevance is the most important motivational 
component. Keller, however, asserts that all four components are necessary, but one or two may be emphasized more 
based on a thorough learner and content analysis. Further research needs to be done to determine if one component is 
more essential to motivation than the others. 

Most of the designers in this study designed instruction for classroom delivery. Since a greater emphasis is 
being placed on designing for technology, it would be interesting to replicate this study with a sample population of 
designers who design primarily for computer or web-based delivery. 

Most of the practitioners said that they received little formal training in design of motivational strategies. 
Further research is needed into the university programs to determine whether or not instructors are including 
motivational design in their course content. Further, it would be helpful to try to determine the factors that influence 
their decisions. 

This study found that the use of an ISD model had a negative influence on the perceived success of the 
project. This conclusion was consistent with studies about the gap between theory and design practice, but it is 
inconsistent with ISD theory. More research needs to be done on how designers make decisions and how they use 
ISD models as well as the factors that influence their choices. 

Finally, this survey study depended on self report rather than on direct observation of designers. It would be 
interesting to study the actual practice of designers and their products to determine the motivational components they 
incorporate into the design of instructional materials and the reason for their decisions. It is important for the field to 
continue to study the gaps between theory and practice and the possible reasons for the gaps. And it is important to 
try to determine the factors that lead to a project’s success. 
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